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ABSTRACT 

The Wasatch Institute for Research and Evaluation 
(WIRE) evaluated the Economic Education in Utah Schools program to 
aid future implementations of the program curriculum materials. This 
evaluation includes: (1) responses to an evaluation survey of 
participants at a Utah State Economics Curriculum Conference at Brian 
Head, August 12-15, 1986; (2) an examination of related program 
documents; (3) consultation with program personnel and program 
documents to revise the final report; (4) an evaluation of the 
dissemination workshop at the annual Utah Education Association 
meeting; and (5) a review of the process and products of the program 
by an Instructional development expert. Results, including 
implementation progress in classrooms as well as local workshops* 
dissemination of materials, are summarized in this report. Extensive 
attachments provide detailed findings from the survey, the survey 
instriiment and procedures, and the full report of the expert review. 
Recommendations are presented for improving both the implementation 
and development processes. These include coordination of workshops 
through key personnel in each school district, additional funding for 
district workshops, more contact between local curriculim leaders and 
the state economics curriculum specialist, better use of the most 
talented and experienced curriculum developers, and adherence to 
specific guidelines for developing instructional materials. (JGL) 
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Evaluating the Implementation of the 
Economic Education Curriculum in Utah Schools 



The Wasatch Institute for Research and Evaluation (WIRE) conducted a 
third-party evaluation for the Implementing Economic Education in Utah 
Schools project This study was requested by the Jordan School District and 
tha Utah State Department of Education* Dr* Bill Quinn was the principal 
investigator for the study. 



Evaluation Goals 

The purpose of the evaluation was to obtain information for improving 
the future implementation in Utah of the Economic Education program 
curriculum materials. More specifically, the following goals were addressed in 
the study: 

L Describe implementation activities which had occurred, or which are 
planned to occur, in school districts that sent participants to the August 
12-15, 1986, Utah State Economics Curriculum Conference at Brian Head* 

2. Survey these participants regarding their opinions of what factors 
promoted greater use of the materials. Ask about their perceptions of 
the curriculum materials, the dissemination workshops, and conditions in 
the participating districts which promoted greater use of the materials. 

a. Materials v/ere examined in the survey regarding: 

L Qarity, scope and applicability of objectives to the curriculum 
needs of districts 

2. Match of materials to objectives 

3. Inclusion of an adequate range of curriculum materials 

4. Transportability of materials to the districts 

5* Availability of materials to the districts at a reasonable cost 
and effort 

b. Workshops were examined in the survey regarding: 

1. Clarity of presentations 

2. Appropriateness of presentation activities for participant needs 

3. Enabling participants to arrange for adequate inservice teacher 
training within tfieir home districts 

c. District participants were also asked to identify factors in their 
districts that promoted implementation of the curriculum materials, 
including such factors as: 

L Availability of trainers with adequate expertise and time to 
conduct the training 

2. Administration support for the training and implementation of 
curriculum materials 

3. Availability of resources in the district or the community to 
implement the curriculum changes in the classroom 
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3. Observe and critique the training workshop conducted in Salt Lake City 
at the annual UEA meeting. 

4. Have an instructional designer review and critique the economics 
education curriculum materials produced by Alpine and Granite districts. 

5. Based upon the above information, the evaluator will reconmiend possible 
improvements in the dissemination strategy to improve the implementation 
of the Economics Education curriculum materials. 



Evaluation Activities 

The experiences and opinions of district participants in the Brian Head 
dissemination workshop were the main sources of information in the study. A 
telephone survey was conducted of all participants in the August workshop. 
This survey sought information regarding the first three goals of the study. 
As appropriate, a follow-up telephone survey was conducted in January, 1987, 
to identify progress made in implementation activities or plans. 

The study included examination of related program documents including a 
summary statement of project activities for Jordan District, a one page 
statement of the law funding the project-Article 53-7-18(l)~documents 
describing the program grants for Jordan, Granite, and Alpine Districts, and 
other documents. 

Program personnel in Jordan District, the pix)gram officer in the Utah 
State Office of Education, and program materials and documents were consulted 
to refine evaluation issues, guide development of instruments, and revise the 
final report 

Dr. Quinn attended the dissemination workshop held at the annual UEA 
meeting in Salt Lake City. He observed the presentation and recorded 
suggestions for improving future workshops. 

An expert review of the products and process used in developing the 
Economics Education curriculum materials was included in the economics 
curriculum implementation evaluation. An expert in instructional development. 
Dr. Larrie Gale, reviewed the process and products from the project. 



Survey Findings 

The following section discusses the findings for the first three evaluation 
goals. These results were obtained from a survey of twelve participants in the 
Utah State Economics Curriculum Conference at Brian Head. All forty school 
districts in the state of Utah were invited to attend the Economics Curriculum 
Conference at Brian Head. Eleven districts responded to the offer of training, 
with five of the eleven districts sending more than one person. Twelve people 
were surveyed for this report; one district (Beaver) did not respond, and two 
districts (Nebo and South Sanpete) had two respondents each. Eleven of the 
twelve respondents attended all four days of the Conference; one person 



attended three of the four days. 

The twelve people who were surveyed were contacted in the month of 
November. They were contacted again in January, 1987, for further 
information on workshops planned or held in their respective districts. 

Findings are presented under the headings of the evaluation goals and 
sub-goals addressed by the survey* The detailed findings from the survey are 
in Attachment 1. The survey instrument and procedures are in Attachment 2. 



1. Describe implementation activities which had occurred, or which are 
planned to occur, in school districts that sent participants to the 
August 12-15, 1986, Utah State Economics Curriculum Conference at 
Brian Head. 

Classrooms 

Many, if not all, of the teachers who went to the workshop at Brian 
Head have incorporated the new Economic Strand materials in fiieir own 
classes with success. The Economics material and worksheets have becpme a 
practical, workable part of their own class work. The teachers of elementary 
students felt that economic principles, even some of the more difficult ones, 
had been understandable to their students. 

Local Workshops 

At this time only two of the eleven districts represented have presented 
workshops for local teachers. Four individuals have prepared in some way to 
hold a workshop but had not finalized plans with their districts. The 
remaining six districts had not discussed workshops or had no plans to hold 
one. 

The financial demands of such workshops was the primary reason given 
for the lack of action on this issue. The $600 provided by the state program 
was not seen as enough funding to organize and promote attendance. Districts 
had already assigned budgetary priorities, and either were not making 
adjustments for this program or were too limited in their discretionary funds 
to consider additional activities. A few districts had a limited budget or none 
at all for social studies. 

Secondary reasons for the lack of implementation were: previously 
established, higher priorities for personnel; lack of trained teachers; and the 
possible inflexibility of teachers. A few districts mentioned having several 
existing programs and therefore they felt an unwillingness to launch another. 
Three people who attended Brian Head said that they did not jfeel competent 
to teach a workshop on their own. A few people expressed concern over the 
possible dearth of teachers who would be willing to learn the economics 
content. 



Local Disseminarion of Materials 



One district, Alpine, placed materials in a central lending library for use 
by all teachers* Alpine also has an interlibrary loan system in place, and 
makes the materials available through that channel Granite District has a 
social studies leader and a department chairman in each school, a system which 
allows easy dissemination of information to teachers. The remaining nine 
districts do not have a system in place* Overall, newsletters, memos, 
handouts, and new curriculum outlines have not been widely disseminated. 

Follow-up S urvey in January. 1987 

All of the districts were surveyed again in late January, 1987, to leam of 
additional implementation acnvides that may have occurred in their districts. 
Of the districts surveyed, no additional workshops were held since the last 
survey. One district. Salt Lake City, has a planned workshop on Feb 24, and 
three other districts are planning to have them but they have not set a 
specific date. The other districts have not had a workshop and have no plans 
for holding one. 



2 . Survey workshop participants regarding their opinions of what 
factors promoted greater use of the materials. Ask about their 
perceptions of the curriculum materials, the dissemination workshops, 
and conditions in the participating districts which promoted greater 
use of the materials. 

The clear, simple, straight-forward program materials were seen as one of 
the strengths of the workshop. One respondent stated that even the concepts 
which he thought difficult to teach and for students to understand have been 
made understandable and easy to use, and some reported favorable student 
responses. The worksheets were easy to adapt and the activities fit varied 
needs; the handouts were particularly well liked by most participants. One 
person did say that there were too many materials and that they should be 
more condensed. 

The State Economics Curriculum Conference was well organized and 
conducted, with excellent presentations. Most of the attenders valued the 
meeting very much. Only one person said it could be shortened to three days 
without losing significant content One person had been interested in bringing 
his spouse, but had not been able to obtain information about that possibility. 
One person said that it did not seem as well attended as it should have been. 

The district which had the resource library and inter-library loan system 
benefited in this program from having a system for dissemination already in 
place. One district was small and could contact appropriate teachers easily, 
and one district had a social studies leader designated to make such contacts. 
One person mentioned having the freedom to add curriculum materials made his 
job easier. 

Most of the participants in the workshop said that their district gave 
them support, but did not have the finances to contribute much to the 
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economics program. Most of the local districts do not seem wilDng to 
underwrite the workshops needed for implementing the curriculum. In several 
districts no discussions have even been held regar<&ig the need for workshops. 



2a. Materials were examined in the survey regarding: 

L Clarity, scope and applicability of objectives to the 

curriculum needs of districts 
2. Match of materials to objectives 
5. Inclusion of an adequate range of curriculum materials 

4. Transportability of materials to thi districts 

5. Availability of materials to the districts at a reasonable 
cost and ^ort 

Only half of the participants indicated that the objectives were clear and 
understandable; three said "somewhat clear" and three did not know or made 
no comment Eight of these people said that the objectives matched Ae 
needs of their districts very well. One said they matched "somewhat", and 
three did not express an opinion. 

Most of the participants thought that the strategy sheets matched the 
economics strand of the state core curriculum. Everyone felt that the new 
economic program included an adequate range of curriculum materials for each 
grade level. 

The majority of participants felt that the curriculum materials were 
available to them at a reasonable cost and effort Two did not think that this 
was the case, and three were uncertain. Most felt that conference materials 
were very easy to obtain and duplicate. However, the cost of the text boclcs 
and the films was mentioned particularly as being more than the home districts 
could afford. 



2b. Workshops were examined in the survey regarding: 

1 . Clarity of presentations 

2. Appropriateness ofprejentation activities for participant 
needs 

3. Enabling participants to arrange for adequate inservice 
teacher training within their home districts 

Participation at the Brian Head Conference 

The participants were very positive in their assessment of the Conference 
overall. Ten of tiie people said that tiiey felt very positive and that the 
conference presentations were very clear. Two said they felt "somewhat 
positive" and that the conference presentations were somewhat clear. They 
said that the Conference was well planned and executed and that they 
particularly liked the handouts which served as study aids and usable materials. 



The activities were appropriate for the information. Most of the 
participants felt that the activities were appropriate; however, four people 
judged them to be "somewhat appropriate." Most of the participants were 
satisfied with the range of information which they had been given. 

There was a range of comments regarding the topics covered at the 
Conference. One person felt that the program should have included more 
information and emphasis on consimier economics. One person sorted through 
the materials to find information he valued. One teacher particularly liked the 
presentation of all elementary level information in one training program. That 
teacher had been changed in liis grade level in the fall and appreciated having 
the training that would fit the hi^er grade teaching situation. 

Most of the people attending the conference felt that they gained enough 
information teach workshops in their own districts. Only three participants 
felt otherwii V, as a result of being unsure of their own abilities and the 
availability of outside help to run a workshop for the other teachers. 

The majority felt that they had been given adequate information to 
arrange for outside trainers. Tluree of the participants did not agree, and two 
of those were concerned about the willingness of such people to come to their 
districts. 



2c. District participants were aiso asked to identify factors in their 
districts that promoted implementation of the curriculum materials, 
including such factors as: 

L Availability of trainers with adequate expertise and time 

to conduct the training 
2. Administration support for the training and implementation 

of curriculum materials 
5. Availability of resources in the district or the community 

to implement the curriculum changes in the classroom 

The districts which implemented the program had several resources which 
enabled them to do so. First of all, the districts which had implemented 
training had been the ones which were involved in the development of the 
materials. The districts also had dissemination facilities in place, and this 
program was easily distributed through these systems. 

The districts which implemented most of the program had personnel who 
were comfortable with the topic. However, most of the districts did not 
mention having trained personnel who could help with this subject area, and 
only one mentioned use of community resources. 

Of those who were surveyed, seven said that their district very strongly 
supported the program. One district held a training conference to reinforce 
the Brian Head seminar. They commented that their districts positive 
supported the program as long as it did not cost the districts to implement the 
program. Two people said that their district did not support the program, 
both because of other priorities. 
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Observation ofUEA Workshop 



A presentation about the new economics curriculum materials was made at 
the annual UEA meeting in the S^lt hrd'ict in Salt Lake City- Invitations and 
announcements regarding the meeting had been sent earlier to the districts in 
the state. Box lunches for teachers who said they planned on attending were 
ordered as an incentive lor coming* Approximately 40 people came to the 
presentation (it seems this was about ha& the number expected based on the 
unclaimed lunches). The presentation lasted firom 12:30 to 2:00. The room it 
was presented in was on the second floor of the convention center out of the 
main traffic path. The room was set up for a group of about 100 to 150 
people. The size of the room and the noise from the air conditioning made it 
harder to hear all of the presenters. 

The purpose of the presentation seemed to be: (1) to provide an 
introduction to the project, explaining its history and relationship to state 
curricular standards; (2) inform attending teachers of implementation workshops 
that their districts might schedule and encourage them to have their districts 
schedule a workshop; (3) present typical lessons from the curriculum materials 
to demonstrate its content; (4) pass out curricular materials (all for grades 7 
to 12, sample materials for grades K to 6); (5) answer questions about the 
materials or the support avaHable for implementation. 

An overview presentation was made by Maurice Capson, Economic 
SpeciaUst from the Office of Curriculum and Instruction, Utah State Office of 
Education. Presentations were made about program development, materials, and 
implementation assistance by the project staff from Alpine and Granite 
Districts. This portion of the program seemed to go fairly well. 

The offer to provide local workshops was repeatedly made during these 
presentations. A clear explanation was made of how to arrange for workshops, 
what costs would be covered by the project and the district, and who teachers 
should contact for more information. However, all of the information on the 
project and the workshops was presented orally, putting a burden on the 
audience to write down the correct information. Perhaps because of the noise 
problem in the room, it was necessary for attending teachers to ask several 
times for workshop information to be repeated. It would have been better to 
provide a handout giving the description of the project, why the districts 
should want to adopt the materials, and how the materials and workshop could 
be arranged 

It was not clear to me timt many of the participants planned on setting 
up a workshop in their district. Attendmg teachers seemed more interested in 
coming first to workshops in other districts to see what would be offered 
before committing to arrange a workshop in their own district. This seemed 
to be a reasonable strategy given the limited resources most districts have for 
such activities. 

Following the informational presentations, several example lessons were 
given by teachers. The demonstrations covered lessons both at the elementary, 
and secondary grade levels. There were some problems with the example 
lessons. The example lessons presented were of uneven quality. Some were 
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well planned an^ skillfully presented; others were just briefly ovendewed by 
the presenter in zn uninteresting style. The lesson handouts also varied in 
quality. A pre-sa^eening of the presentations and presenters would have made 
the experience belter. 

In spite of the uneven presentations, participants seemed interested in the 
materials. Sevefal asked questions, particularly on how to get more 
information, help with adopting, etc. Many of die participants took a full set 
of handouts witii tiiem. Those I talked to said this was die first opportunity 
tiiey knew of for learning about die materials, not being aware of the Brian 
Head workshop until they were told in this meeting. Mosi seemed to feel the 
materials would be relevant to thdr setting. 



Summary of Expert Review of Materials 

In addition to surveying participants and observing training workshop, a 
review was made of die curriculum materials by an instructional designer. The 
full report of the expert review is in Attachment 3. The main points of that 
review are summarized here. 

Objectives 

The objectives of die state seem to be two-fold: (1) to teach die basic 
principles of die free enterprise system; and (2) to motivate support of diat 
system* The materials are written as exercises in learning defined concepts. 
That method works well widi die transfer of information and development of 
definitions, which was die first goal. However, it is not die mediod of choice 
for motivating behavior and active support for die free enterprise system, 
which was the second goal. Other instructional approaches are recommended 
for changing die attitudes and behaviors of smdents. 

Learning Types 

Learning tasks may be divided into categories, and the instruction may 
then be planned for each differing category and its requirements. The 
information on economics could be presented as discrimination learning or as 
defined concepts. Instruction for these two types will necessarily require 
different approaches; neither is aimed at motivation. 

General Problems 

1) Several objectives appeared to be very ambitious and the instructional 
materials, dierefore, could not readily measure up to that which was 
required. 

2) Some classroom activities and simulations may act as distractors. These 
activities can be improved by emphasizing the basic principles being 
taught rather than details or mechanics. 
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3) 



Goals were not always we^I matched with classroom activities, simulations, 
or other suggestions. 



4) Many of the materials used in the development of the program were 
apparently taken from published sources. Copyright violation is a 
possibility in tliose cases, and ought to be examined. 

5) Variation between grade levels and development teams seemed to be the 
result of uneven abilities between Lidividual developers. Some people 
consistendy developed better materials than did others; they were better 
able to match activities to objectives and provide a unified concept 



Conclusions and Recommendations 

From the foregoing information it is clear that much good work has been 
done in developing and distributing the new economics curriculum program in 
Utah. The goals and basic elements of the economics curriculum project are 
valued by teachers. Generally, teachers feel the curriculum matenals developed 
to date are of good quality and appropriate for their classes. It appears that 
a good start has been made in implementing the new curriculum. However, a 
continued effort is needed to ensure a high level of implementation of the new 
curriculum materials. The study findings also suggest possible changes in the 
development process that could lead to even higher quality in future curriculum 
projects. 

Implementation Process 

The implementation process begun this year represents a good start. 
However, this process will need to be continued and extended to support wide 
use of the ne>y econonucs curriculum materials in Utah. Three points should 
be addressed in future implementation activities. These issues are well 
understood by the project personnel, but local districts have not yet been 
prepared to carry out these steps. 

Contact all kev district personnel. It is important that the program 
personnel meet with key individuals from each school district in Utah in 
meetings like the Brian Head workshop. The people invited should have the 
responsibility in their district for coordinating the social studies cuniculan. 
In larger districts several people may need to be involved* This will probable 
require advanced planning of several workshops to accommodate their 
schedules. 

Resources for district workshops . The major snag this year to 
implementing the new curriculum was the lack of adequate funding in the 
districts to release teacher during the day or pay teachers after hours to come 
to district workshops. The $600 offered to districts was not enough in most 
cases to stimulate action. If districts cannot find the resources to conduct 
workshops, then the implementation of the economics curriculum will be 
severely limited. 
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Perhaps in the coming year districts will h;ivc enough advanced notice 
now to plan the woricshops into their schedule for inscrvice training. 
Consistent encouragement should con*, from the State Office of Education to 
do this. Follow-up material on workshops should make it easier for them to 
plan and conduct woricshops. Also, inviting key personnel fixjm one district to 
attend woricshops in neighboring districts may answer many questions about the 
importance of this implementation activity. 

Continued support. Implementation will require a continued effort in each 
district after the initial workshops. Local curriculum leaders need continued 
contact with the state economics curriculum specialist to maintain tiie 
momentum for implenoentation. Full implementation will require a series of 
activities each year in the districts to help teachers understand and use the 
materials. The state specialist should be contacting these individuds to 
provide encouragement and ideas for implementation. 

D evelopment Process 

A number of changes might be made in the staffing, strategy, and 
management of future curriculum development projects to make them more 
effective. 

Siaffing. Development team members should be chosen so that each one 
is has training and experience in development strategies and the use of 
evaluation findings. Team members should be good writers, as well as 
experienced teachers. Competitive activities could help to identify talented 
and experienced people. The team would benefit fiom the assistance of an 
evaluator who would help them examine design decisions and draft materials 
early in the process. 

.^iatcgy. There are nine steps that should be used in future 
instntctional development project like this. They include: 

1) Need assessment 

2) Task and/or content analyses 

3) Researching available reso^ices and references 

4) Instructional ^stem and strategy design 

5) Development of the instruction^ materials 

6) Formative evaluation of the resulting materials 

7) Revision of the materials, strategy, or system based on the formative 
evaluation results 

8) Implementation of the course 

9) Summative evaluation of the course 

In addition to the above steps we recommend that development teams use 
what we refer to as fast prototyping. That is, we recommend that a strategy 
be developed consistent with learning types identified in tlie content/task 
analyses and that its processes be immediately tested in different school 
settuigs with two or three different lessons. 

Management The state should require approval of development at each 
phase for continued funding. Instructional design documents should be used as 
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a stimulus for thinking through the products and procedures in advance and 
thereby structuring the plannmg and implementation. 
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Attachment 1: Detailed Survey Findings 
[Note: Answers have been grouped together and renumbCTwi, the numbers do 
match the original survey items.] 
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Brian Head Workshop Evaluatio n 



1. DID YOU STAY FOR THE ENTIRE FOUR DAYS OF THE WORKSHOP? 

11 All four days 
1 Three days 



1. WHAT WAS YOUR OVERALL IMPRESSION OF THE WORKSHOP? 

10 Very positive response 
2 Somewhat positive 
0 Not very positive 

WHY DID YOU ANSWER VERY POSITIVE OR SOMEWHAT POSITIVE? 

(a) I liked the fact that the conference was very well planned with 
many useful ideas, many of which Fve already used 

(b) Fm just starting to teach economics and I found the 
information given me at the conference very helpful* 

(c) I received a lot of useful materials-suitcases full, 

(d) The handouts were great; participants could listen without 
taking notes and still have all the information that was 
covered 

(e) The historical perspective is good, but the consumer side of 
economics was not fully covered. 

(f) The goal presentations are prepared and clear, but they could 
be shorter by a full day and still be understood. 



3. HOW CLEAR AND UNDERSTANDABLE WERE THE PRESENTATIONS AND 
ACnvmES AT BRIAN HEAD? 

10 Very clear 
2 Somewhat clear 
0 Not very clear 

WHY DID YOU ANSWER VERY CLEAR OR SOMEWHAT CLEAR? 

(a) They were easy to understand; I could see how to implement 
them in my own classes. 

(b) The materials are relevant; I would recommend no changes. 

(c) The presentations made information I concepts an easy and 
normal part of teaching, including some of the harder concepts. 

13 
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HOW APPROPRIATE WERE THE WORKSHOP ACnVITIES FOR YOUR 
NEEDS AND THE NEEDS IN YOUR DISTRICT? 

8 Very appropriate 

4 Somewhat appropriate 

0 Not very appropriate 

WHY DID YOU ANSWER VERY APPROPRIATE OR SOMEWHAT 
APPROPRIATE? 

(a) I liked the idea of going to a workshop about all grades at the 
elementary level. For the last year I taught second grade, but 
when I came back to school I was switched to sixth, so it was 
important to have knowledge about the fiill range of programs. 

(b) The conference was just what I needed 

(c) Some things are okay. I sifted through materials and tailored 
them to my own needs. 

(d) I think there needs to be a change in the core curriculum in 
the state to mandate an economics class. 

(e) Consumer economics needed more treatment to make the 
curriculum materials more appropriate. 



DID YOUR PARTICIPATION IN THE WORKSHOP ENABLE YQU TO 
PROVIDE ADEQUATE INSERVICE TRAINING TO TEACHERS WITHIN 
YOUR HOME DISTRICT? 

9 Yes 

2 No, but I didn*t expect the workshop to do this. 

0 No, but another person who attended from my district did learn 
enough to do inservice training. 

1 No, and it should have. 

COMMENTS: 

(a) I think for me, yes; but the average teacher who came was not 
already trained in making presentations. For most teachers, I 
would say no. 

(b) The last day was the best-on how to make a presentation in 
your home district But our home district could not put on a 
workshop for $600. 

(c) I can*t do it by myself. I need more background in specific 
implementation technique and strategy. 

(d) I would need more help if I were to put on a workshop. 
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DID THE WORKSHOP PROVIDE ADEQUATE INFORMATION ABOUT HOW 
TO ARRANGE FOR OUTSIDE TRAINERS TO COME AND GIVE INSERVICE 
TRAINING TO TEACHERS WITHIN YOUR DISTRICT? 

9 Yes 

1 No, but I didn't expect the workshop to do this 

2 No, and it should have 

COMMENTS: 

(a) They gave a good offer of help. They were clear about their 
offer with our inservice. 

(b) We were mailed a list afterwards-great list-but money is a 
problem in our district 

('^^ The "how to" is good-it is money that is no! The cost when 
you consider substitutes for the teacher's home classes and 
travel time [is too high J $600 is too litde. 

(d) No, but Fm so new in teaching economics that my own 
resource pool is very limited 



WHAT OTHER COMMENTS OR SUGGESTIONS YOU WOULD LIKE TO 
MAKE ABOUT THE BRIAN HEAD CONFERENCE? 

(a) The facility was very good-much better than expected* 

G)) It was worthwhile. I want a follow-up conference. 

(c) I made friends. I went to the Alpine Conference (not his district) 
to get more information. 

(d) I am not sure if a trainer would travel to my home district for a 
workshop. 

(e) I need help in setting up a workshop! 

(f) I need help with resource personnel in own area. 

(g) Want more coordination between secondary and elementary grade 
levels in materials. 

(h) I don't like die Cedar City College B letter grade received for those 
who wanted credit The B hurts my GPA and it was not clear why 
I did not receive an A. This seemed unfair and arbitrary. 

(i) Why didn't more districts show up? In some cases we needed more 
appropriate personnel than the ones that did come. 
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Economics Materials Comments 



8. WHAT PARTS OF THESE MATERIALS DO YOU OR YOUR DISTRICT NOW 
HAVE? 



10 A copy of the new objectives for the economics strand of the state 
core curriculum. 



5 Full set of elementary grade (K to 6) strategy sheets and materials, 

1 Partial set of elementary grade (K to 6) strategy sheets and 
materials. 



10 Secondary grade (7 to 12) instructional activity sheets and materials, 

1 1 Secondary grade (7 to 12) catalog of instructional materials, 

9 Computer disk versions of the secondary instructional activities and 
the secondary catalog. 

12 Other economic education materials obtained through the Brian Head 
workshop (individual lesson sheets, materials produced by other 
states, etcO 

Note. Most of participants had trouble remembering what materials they 
had without going through them in front of them. Also, a list of those 
wanting some part of the materials cited was supply to the State Board 
of Education staff to aid in providing the wanted materials. 



9. HAVE YOUR READ THE NEW STATE OBJECTIVES FOR THE ECONOMICS 
STRAND OF THE STATE CORE CURRICULUM? 



10 Yes (3 of them said only at the workshop, not since) 
2 No 



10. HOW CLEAR AND UNDERSTANDABLE ARE THESE OBJECTIVES? 

6 Very clear 

3 Somewhat clear 

3 Don't know or no comment 

WHY DID YOU ANSWER VERY CLEAR OR SOMEWHAT CLEAR? 

(a) The basic objectives fit the program ongoing in our district 
now. 



(b) They seem to be very close to tlie needs of students in our 
district. 
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(c) They arc easy to understand; the language is related to 

everyday terms- 
Note: Most participants could nor. remember the objectives separate 
ftom the other materials. 



1 L HOW WELL DOES THE ECONOMICS STRAND OF THE STATE CORE 
CURRICULUM MATCH THE NEEDS AND CONDITIONS IN YOUR 
DISTRICT? 

8 Very well 
1 Somewhat 
3 Don't know 

WHY DID YOU ANSWER VERY WELL OR SOMEWHAT WELL? 

(a) They match what is being used in our district now very well- 
[Two comments] 

(b) It is a great program but it's hard to do in a smaller district 
than the Salt Lake area- 



(c) It fits my needs very well, but I would need a lot of training 
to make the program usable to ether teachers in my district 

(d) I used these materials in my own classroom with success- But 
I would need to teach others about them to make an impact in 
the district Teachers need to have a chance to do Ae 
program, not just pick up information at a workshop- 

(e) It appears that the curriculum is just what we need 

to have students understand about cconomics- 



(f) I have applied the economics principles to social studies- 

(g) The content is well rounded, except as mentioned before- 
[consumer issues] 

(h) The content should be used in some history courses as welL 



12- HAVE YOUR READ SOME OF THE NEW ECONOMICS CURRICULUM 
STRATEGY SHEETS (INSTRUCTIONAL ACnVTTY SHEETS) AND THEIR 
ACCOMPANYING MATERL\LS? 



11 Yes 
1 No 
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HOW WELL DO THE STRATEGY SHEETS (INSTRUCTIONAL ACnVTTY 
SHEETS) MATCH THE OBJECTIVES IN THE ECONOMICS STRAND OF 
THE STATE CORE CURRICULITM? 

10 Very well 
1 Somewhat well 
1 Don*t know 

WHY DID YOU ANSWER VERY WELL OR SOMEWHAT WELL? 

(a) I like the strategy sheets the best and am using them with my 
sixth grade class. 

(b) The activities are great 

(c) The curriculum should be revamped to cover all th 5 materials 
from the text book, 

(d) I have no changes to suggest 

(e) They are being used in the district now, in most -r^^.sts, except 
without the formal training. 

(f) They are very close. 

(g) I answered "somewhat well" because I need to see what would 
work for me. We could use all the ideas in the worksheets. 



DOES THE NEW ECONOMIC PROGRAM INCLUDE AN ADEQUATE RANGE 
OF CURRICULUM MATERIALS FOR EACH GRADE LEVEL? 

12 Yes 
0 No 

COMMENTS: 

(a) They build on each other very well to make a complete 
program. 

(b) They are adequate, with good starting level activities. Of 
course I would like to see more materials provided but the ones 
that were are adequate for getting the program started. 

(c) It is good to see the total program so we can work toward 
common end goals. 

(d) Training for all teachers must be done on a larger scale if they 
hope to have all teachers using the program. 

(e) Again, I would like to see more consumer level materials in 
curriculum. 
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(f) They are generally good, but I would like to see more on the 
7th to 8th grade level to match the 10th through 12th grade 
materials. 

(g) Getting the teachers to implement the program will be the 
problem, especially if they are ready to retire, or if this 
subject is out of their area- 
Five people had no comments. 



Aspects of the Materials Influencing Implementation 



15. ARE THERE ANY ASPECTS OF THE NEW ECONOMICS CURRICULUM 
MATERIALS THAT WE HAVE NOT DISCUSSED THAT MAKE IT EASY TO 
USE THEM IN YOUR DISTRICT? 

(a) Eight people said "No" to this question, and one said he could not 
answer. 

(b) The program is good as it is. 

(c) All of it is easy to use-motivation to do 5 j is what is needed now. 

(d) There is a problem with having very few economics classes being 
taught, so the materials get only slight use. 



16. OTHER THAN WHAT WE HAVE DISCUSSED, WHAT ASPECTS OF THE 
NEW ECONOMICS CURRICULUM MATERIALS MAKE IT DIFFICULT TO 
USE THEM IN YOUR DISTRICT? 

(a) Lack of money is the main problem. [Two people] 

(b) Resources are limited. There is no computer system to use in our 
district as there is in large ones, and there are no other trained 
personnel to help with implementing the program. 

(c) The text and film are too costiy and we can't afford them in 
amounts large enough to fill our needs. 

(d) Teachers are reluctant to change. They resist change and this 
resistance is always a factor in implementation of new materials. 

(e) We need more grade level specific materials showing how to use the 
materials at different grade levels. 

(f) My problem is that I am a new teacher. Talking to older teachers 
is a problem, because of fear of being accepted. Economics is also 
very new among the teachers in the subject area in our district. 
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(g) Our district is not paying much attention to this program. The lack 
of money is a big problem* 

(h) One problem is that to use full impact or to provide enough 
coverage, an entire class in economics should be offered With the 
Sta*j& Core Requirement for World Geography, economics must be 
offered as an elective only* 



District Factors Influenc ing Implementation 



l§r ARE THE NEW ECONOMICS CURRICULUM MATERIALS AVAILABLE TO 
YOUR DISTRICT AT A REASONABLE COST AND EFFORT? 

7 Yes 

2 No 

3 Don't know 
COMMENTS: 

(a) The materials are located in three different places in the 
district and can be checked out, which makes them very 
accessible. 

(b) We sent for and received materials at reasonable cost 

(c) Copies are easy to get by xeroxing them. 

(d) I need to know what lo get; the workshop helped with that I 
have been able to get the materials when I ask for them 
through the district office. 

(e) The materials are very cheap, and they can be tied into the 
social study text we ^eady have. They could do it, but the 
district is not spending for them. 

(f) We have a limited budget; the text is okay, but we don*t have 
money for the film strip. Our limited district budget does not 
allow for expenditures at this time. 

(g) The program is very expensive. The State Office has provided 
materials that were needed. 

(h) The cost is too high! The text book and special items are easy 
to locate but funding their purchase is harder because of 
cutbacks from the state legislature. 

(i) Money is a problem in our district 
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(j) Three people did not know about the costs or availability of 
materials. One of the three knew that the budget was zero for 
social studies materials. 

Note: Most felt conference materials were very easy to obtain and 
duplicate, but text and film strip were too expensive. 



WHAT FACTORS IN YOUR DISTRICT MAKE IT EASIER TO IMPLEMENT 
THE NEW ECONOMICS CURRICULUM MATERIALS? 

(a) We have a strategy bank to draw fh)m in the district which includes 
a whole room with files of resource materials to use. 

(b) There is administrative flexibility in design of materials that are 
used to cover curriculum requirements. 

(c) The small size is an advantage. It is easy to reach all of the 
teachers in the district with responsibility in this area. 

(d) I have firedom to implant material in lesson plans. Fm not sure 
about other teachers in the district other than my own situation. 

(e) The social studies books can fit into the economics curriculum with 
a minimum of effort 

(f) Well prepared teachers and good text books. 

(g) The district personnel are supportive but the lack of money for 
economics education is a very limiting factor. 

(h) They have been supportive, but we have not discussed a workshop 
yet 

(i) We are beginning and heading in the right direction. We are having 
a meeting in the district to set up the social studies curriculum. 
They are pushing the economics program. 



WHAT FACTORS IN YOUR DISTRICT MAKE IT MORE DIFRCULTT O 
IMPLEMENT THE NEW ECONOMICS CURRICULUM MATERIALS? 

(a) I need more aid (personnel) in District to help me set up workshop. 

(b) Money is a problem; there is no funding for a workshop. 

(c) Teachers are apathetic, settied in an old routine and resist change. 

(d) There has been no formal training in economics for teachers. 

(e) We have a lack of resource material, especially films and filmstrips. 
[Two people] There is a lack of funds. 
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(f) There arc only two teachers in the district in social studies. 

(g) The resource bank is in American Fork, which is too far for many 
school teachers in Qrem to use easily* 

Three people said that there were no factors which restricted 
implementation. 



HOW STRONGLY DOES THE DISTRICT ADMINISTRATION SUPPORT THE 
TRAINING AND IMPLEMENTATION OF CURRICULUM MATERL\LS IN 
ECONOMICS EDUCATION? 

7 Very strongly 

4 Somewhat strongly 

0 Not very strongly 

1 Don't taiow 

miY DID YOU ANSWER VERY STRONGLY OR SOMEWHAT 
STRONGLY? 

(a) We have had a training conference to remforce the Brian Head 
seminar* 

(b) The district has helped to set up workshops. 

(c) The supervisor backs me, including with funding. 

(d) They give us what we ask for as long as money is available- 
which is getting to be a large problem. [Two people] 

(e) The district is gung-ho for it but they don't have funding for 
inscrvice or special materials. 

(f) They back us-as long as it doesn't cost 

(g) There are no funds available for the program. [Two people] 

(h) There is not much interest from the district staff for this 
program. [Two people] 

(i) We have not planned any follow-up training. 

(j) Our district is busy with converting to a trimester system. We 
have not talked about a training program, but when and if we 
hold one, I will need some help. 
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2L WHAT KIND OF RESOURCE PEOPLE DO YOU HAVE IN THE DISTRICT 
OR THE COMMUNITY TO HELP IN IMPLEMENTING THE NEW 
ECONOMIC CURRICULUM IN THE CLASSROOM? 

(r) Wc have an intcrlibrary loan system through the media center. 

Qo) We have a social studies leader in one district, with a department 
chairman and a program to bring in local community and business 
leaders. 



(c) None-^Fm the main resource person. 

(d) In the district Tm not sure. The community is wide open! 

(e) Letters firom the banker and other materials firom the Federal 
Reserve. 



(f) The bank is willing to help, providing fiee check books for students 
to use for /practice. The crcdt union has pamphlets on banking 
functions. 



(g) Various local business people arc available and willing to help. 
[Four people] 



22. WHAT OTHER COMMENTS WOULD YOU LIKE TO MAKE ABOUT THE 
CONDITIONSINYOURDISTRICr FOR IMPLEMENTING THE NEW 
ECONOMICS CURRICULUM MATERL\LS? 

(a) We need more money to implement this workshop and program. 
[Two people] 

(c) Wc have so much going on in the district we may pull back from 
the workshop program for the time being. There are so many 
programs, and there are not many people who push economics. 

(d) Fve found it hard to work with my district coordinator on this 
project. 

(e) We have an intcrlibrary loan system through the media center, which 
makes it easier to obtain needed reference and resource materials. 



(e) No comments. [Seven people] 
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District Implementation Activities 

23. WHAT TRAINING WORKSHOPS HAVE BEEN HELD FOR TEACHERS OR 
ADMINISTRATORS IN YOUR DISTRICT TO DISCUSS THE NEW 
ECONOMICS EDUCATION I/IATERIALS? 

Only 1 workshop has been held at time of survey. 

For this workshop, two teachers from each school in district were invited 
to participate in a 4 hour workshop. If: was held on the third Thursday 
in October in the evening. Teachers woe provided with curriculum 
materials, including worksheets so they could apply the new fxonomic 
strand in their classrooms. A follow-up district workshop will be held in 
January or Februar/, 1987. 



24. WHAT ECONOMICS CURRIOJLUM MATERIALS OR OTHER RESOURCES 
HAVE BEEN MADE AVAILABLE TO TEACHERS IN YOUR DISTRICT WHO 
DID NOT ATTEND THE BRIAN HEAD WORKSHOP? 

Materials for non-attending teachers were not generally distributed to 
teachers but they were available if requested or shared on a one-to-one 
basis. 

25. WHAT ORIENTATION MEETINGS HAVE BEEN HELD FOR TEACHERS OR 
ADMINISTRATORS IN YOUR DISTRICT TO SHOW THEM THE NEW 
ECONOMICS EDUCATION MATERIALS? 

Two districts have held formal orientation meetings. In the oilier 
districts only brief, informal conversations with district personnel were 
reported. 



26. WHAT OTHER MEANS, IF ANY, HAVE BEEN USED FOR INFORMING 

TEACHERS ABOUT THE NEW CURRICULUM MATERL\LS (NEWSLETTERS. 
FACi JLTY MEETING ANNOUNCEMENTS, ETC.)? 

There was no other means being used to inform teachers about the 
economics materials. 



27. WHAT ACTIVITIES ARE PLANNED FOR THE NEXT FEW MONTHS TO 
IMPLEMENT THE ECONOMICS EDUCATION MATERIALS IN YOUR 
DISTRICT? 

(a) We did one workshop and I don't know when another one is planned. 

(b) November 2nd and 18th~(two Tuesdays in November) are planned for 
our meeting with all social studies teachers in the district. 

(c) On Jan. 17th a meeting is planned for all social studies teachers in 
the district 
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(d) Fvc planned or have plans for a workshop but the district has taken 
no action to help or set a date as of right now* 

(e) Workshop dates are up in the air, maybe next spring. 

(£) None right now-maybe at end of January, first semester. The 
funding of $ 600 is not enough and the district has no plans for 
funding. 

Note: These are the only positive responses to workshop plans. The six 
other district personnel did not mention any plans at all for a workshop. 



Final Comments 



28. ARE THERE ANY OTHER CHANGES THAT SHOULD BE MADE IN THE 
TRAINING, THE MATERIALS OR IN HOW THEY ARE PROVIDED TO 
TEACHERS SO THAT THE CURRICULUM WILL BE MORE USEFUL? 

(a) This is a good program. It needs time to really blossom into a full- 
fledged program. 

(b) I would like more help in workshop preparation, more training with 
more district personnel trained and involved 

(c) Three types of workshops should be planned for different interest 
groups: Elementary (K-6); Junior High (7-9); and High School 
(10-12) 

(d) Don't divide elementary teachers in such small groups* They need 
more general, overall coordination, and information for all 
elementary grade levels* 

(e) We have not yet addressed how funding may be found on district 
levels. 

(f) We must have feedback on those in the state who are doing the 
program. 

(g) A lot of information was given-a big job. Be more specific. Distill 
it down into a more manageable form. It would help to set goals 
and get started doing the program. 

(h) Collect materials in standard-size paper with holes punched for 
inclusion into a binder. That would make it easier to organize and 
keep information. 
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Attachment 2: Survey Form and Procedures 
[Note: Blank space for recording responses removed to reduce 




WIRE Survey of Implementing Economic Education in Utah 



FormID# 

Name & Title 

Phone # ( ) - ^extensioa 

Mailing Address , 



Survey Litrodudion 

Hello. Is this ? 

Thisis , and I am calling ftom Salt Lake Gty- TheUtah 

State Office of Education has asked my group to do a state-wide survey to 
find out what teachers thii \ of the new economic education materials and 
training being provided in Utah this year. The results of the survey will be 
used m planning improvements in both the material and in the inservice 
training of Utah teachers. 

I understand that you attended a workshop in August at Brian Head to learn 
about the new curriculum material. Is that correct? [IF THE PERSON DID 
HOT ATTEND THE BRIAN HEAD WORKSHOP, ASK IF THEY KNOW WHO 
FROM THEIR DISTRICT DID.] 

The questions that I need to ask should take about 20 minutes. I want to 
add that I would be happy to answer any questions you might have about the 
study, either now or later. Okay? 

To start off with, I would like to ask you about the Brian Head workshop. 
L Did you stay for the entire four days of the workshop? 
[ ] yes 

[ ] no How many days were you there? 

2. . What was your overall impression of the workshop? [READ OPTIONS] 

[ ] very positive 

[ ] somewhat positive 

[ ] not very positive 

[ ] don*t know, couldn't say [DON'T READ THIS OPTION] 

3. Why did you answer very positive/somewhat positive/not very positive? 
(PROBE: What changes were needed to make it more positive?) 

4 How clear and understandable were the presentations and activities at 
Brian Head? [READ THE OPTIONS] 

[ ] very clear 

[ ] somewhat clear 

[ ] not very clear 

[ ] don't know, couldn't say [DONT READ THIS OPTION] 
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5* Why did you answer very clear/somewhat clear/not very clear? 
(PROBE: What changes were needed to make them clearer?) 

6. How appropriate were the workshop activities for your needs and the 
needs in your district? (READ THE OPTIONS] 

[ ] very appropriate 

[ ] son^what impropriate 

[ ] not very appropriate 

( ] don't know, couldn't say [DONT READ THIS OPTION] 

7« Why did you answer very appropriate/somewhat appropriate/not very 
appropriate? (PROBE: Was the scope of the presentations appropriate? 
Was the instruction well matched to your knowledge of economics? Was 
the content of the instruction relevant for the students in your district?) 

8. Did your participation in the workshop enable you to provide adequate 
inservice training to teachers within your home district? [READ 
OPTIONS] 

[ ] yes 

( ] no, but I didn't expect the workshop to do this* 

( ] no, but another person who attended from my district did leaiii 

enough to do inservice training. [BF OPTION SELECTED ASK: Who is 

tfiis? ] 

( ] no and it should have. [IF THEY SELECT THIS OPTION ASK: 

What changes should have been made in the workshop to prepare 

you to do inservice training? 
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9* Did the workshop provide adequate information about how to arrange for 
outside trainers to come and give inservice training to teachers withki 
your district? [READ OPTIONS] 

[ ] yes 

[ ] no, but I didn't expect the workshop to do this. 

[ ] no and it should have. [IF THEY SELECT TfflS OPTION ASK 

What changes should have been made in the workshop to better do 

this?] 

10* What other conmients or suggestions you would like to make about the 
Brian Head conference? 



Now, I would like to ask you about the new economics education curriculum 
materials. 

1 1. What parts of these materials do you or your district now have? [READ 
THE LIST. FOR PARTS THEY DONT HAVE ASK: WiU you be getting 
this in the next month or two? IF THEY SAY NO, ASK WHY AND 
WRITE ANSWER IN THE MARGIN BELOW EACH PART.] 



NOW WILL 
HAVE GET 



a. 



b. 



d. 



e. 



h. 



12. 



A copy of the new objectives for the economics strand of the 
state core curriculunL 

Full set of elementary grade (K to 6) strategy sheets and 
materials. (Equals a 2* to 3* thick stack of paper ' vith folders 
for each grade.) 

Partial set of elementary grade (K to 6) strategy sheets and 
materials. 

Which grades do you have (circle)? K 1 2 3 4 5 6 

Secondary grade (7 to 12) instructional activity sheets and 

materials. (1" thick, 272 pages, color coded in a 3 ring binder. 

It is entitled Economics Classroom Activities.) 

Secondary grade (7 to 12) catalog of instructional materials. 

(76 pages, color coded. It is entitied Economics Resources.) 

Computer disk versions of (1) the secondary instructional 

activities and (2) the secondary catalog. 

Other economic education materials obtained through the Brian 

Head workshop (individual lesson sheets, materials produced by 

other states, etc.): 



Have you read the new state objectives for the economics strand of the 
state core curriculum? 



[ ] no [SKIP TO QUESTION 17] 

[ ] yes 

13. How clear and understandable are these objectives? [READ THE OPTIONS] 

[ ] very clear 

[ ] somewhat clear 

[ ] uot very clear 

[ ] don't know, couldn't say [DONT £?EAD THIS OPTION] 
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14. Why did you answer very dear/somewhat dear/not very dear? (PROBE: 
What changes are needed to make them clearer?) 

15. How well docs the economics strand of the state core curriculum match 
the needs and conditions in your district? [READ THE OPTIONS] 

[ ] very well 

[ ] somewhat well 

[ ] not very well 

[ ] don*t know, couldn't say [DON'T READ THIS OPTION] 

16. Why did you answer very well/somewhat well/not very well? (PROBE: Is 
the scope of the objectives appropriate? Arc die objectives well matched 

to the ability levels of the students? Is the content of the objectives 
appropriate for the students in your district?) 

17. Have you read some of the new economics curriculum strategy sheets 
(instructional activity sheets) and their accompanying materials? 

[ ] no [SKIP TO QUESTION 21] 
[ ] yes 

18* How well do the strategy sheets (instructional activity sheets) match the 
objectives in the economics strand of the state core curriculum? [READ 
THE OPTIONS] 

[ ] very well 

[ ] somewhat well 

[ ] not very well 

[ ] don't know, couldn't say [LON'T READ THIS OPTION] 

19. Why did you answer very well/somewhat well/not very well? (PROBE: 
What changes are needed to make them a better match?) 

20* Does the new economic program include an adequate range of curriculum 
materials for each grade level? 

[ ] yes 

[ ] no What additional materials at what grade level might be included 
in them? 

2h Are the new economics curriculum materials available to your district at a 
reasonable cost and effort? 

[ ] yes 

[ ] no How could they be provided in an easier or more convenient 
way? 

22. Are there anj' aspects of the new economics curriculum materials that we 
have not discussed that make it easy to use them in your district? 
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23. Other than what we have discussed, what aspects of the new economics 
curriculum materials make it difficult to use them in your district? 



Now, I would like to ask you about conditions in your district that may effect 
your use of the new economics education curriculum materials. 

24. What factors in your district make it easier to implement the new 
economics curriculum materials? 

25. What factors in your district make it more difficult to implement the new 
economics curriculum materials? 

26. How strongly does the district administration support the training and 
implementation of curriculum materials in economics education? [READ 
OPTIONS] 

[ ] very strongly 

[ ] somewhat strongly 

[ ] not very strongly 

[ ] don*t know, couldn't say [DONT READ TfflS OPTION] 

27. Why did you answer very strongly/somewhat strongly/not very strongly? 

28. What kind of resource people do you have in the district or the 
community to help in implementing the new economic curriculum in the 
classroom? 

29. What other conmients would you would like to make about the conditions 
in your district for implementing the new e*:onomics curriculum materials? 

IF SEVERAL PEOPLE FROM A DISTRICT CAME TO THE BRL\N HEAD 
CONFERENCE, ONLY THE DISTRICT CURRICULUM SPEOALIST IN SOCIAL 
STUDIES (OR A SIMILAR PERSON) WOULD BE ASKED THE FOLLOWING 
QUESTIONS. 

IF YOU DONT ASK THESE QUESTIONS FOR THIS INTERVIEW, SKIP TO 
QUESTION 35. 

30. What training workshops have been held for teachers or administrators in 
your district to discuss the new economics education materials? (PROBE: 
How many meetings have been held? When were they held? How much 
time was spent in the meetings? How many teachers or administrators 
attended? What grade levels (and curriculum areas at the upper grade 
levels) were represented? What resulted finom this activity?) 

3 1 . What economics curriculum materials or other resources have been made 
available to teachers in your district who didJlQLattend the Brian Head 
workshop? (PROBE: Where are the materials housed? How do teachers 
get access to them?) 
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32. What orientation meetings have been held for teachers or administrators 
in your district to show them the new economics education materials? 
(PROBE: How many meetings have been held? How much time spent was 
spent in the meetings? How many teachers or administrators attended? 
What grade levels (and curriculum areas at the upper grade levels) were 
represented? What resulted from this activity?) 

33. What other means, if any, have been used for informing teachers about 
the new curriculum materials (newsletters, faculty meeting announcements, 
etc)? (PROBE: To which teachers was this information distributed? 
When? What resulted from this activity?) 

34. What activities arc planne4 for the next few months to implement the 
economics education materials in your district? 



Type of activity Date planned? Who will attend? 

Informational 

meeting: 



Teacher workshop: 
Other: 



35. Are there any other changes that should be made in the training, the 
materials or in how they arc provided to teachers so that the curriculum 
will be more usefiil? 

36. Aic there other in your district that we should also call to learn more 
about the use of the new economics education material in your district? 

Name & Position:^ 

How to locate: 

(get just enough to contact, don't bo^; down) 

Name & Position: 

How to locate: 

Name & Position: 

How to locate: 



Thank you for your kind assistance! You have been very helpfuL 
[Remember to record interview time and to check all answers.] 
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Rule Book for Economics Education Surve y 



A* Who to talk to: 

You want to interview the participants at the Brian Head workshop held 
this August (sec attached list). 

If time permits and the Brian Head participa nts from a district are not 
district curriculum leade rs in economics , you may also interview the 
district social science curriculum speciaUst or someone else identified as 
being a key district person for economics curriculum. 

Do not spend a lot of time interviewing or discussing the survey with 
individuals who do not qualify for the target group. 

B. Before you start . . . 

Make sure you have the forms, at least three sharpened pencils, and a 
clear working area for writing. Make sure the work place is quiet and 
that it will stay that way for the duration of the interview. Make sure 
your throat is clear sounding. If you have a cold, take a decongestant or 
something to sound normal. Have a glass of water available to help you 
if you get a frog in your throat during an interview. 

C. For the interview be sure to . . . 

1. Mark the time the interview starts on the cover sheet 

2. Read the questions clearly without giving suggested answers. 
Memorize the questions and the layout of the form so you can read 
questions without hesitation and quickly find the proper place to 
record answers. 

3. If it seems that a question has been misunderstood, do noLtell the 
respondent that you think he misunderstood. Instead, these responses 
may be of help. 

Could I read the question and the answer Fve written down 
just to be sure I have everything you wanted to say. 

I think I may not have read the question correctiy, so, may I 
read it again to be sure. 

4. Use neilfiaLpi'obcs as needed. If you are in doubt about how to 
interpret the respondent's answer or what it means, we will be in 
even greater doubt. Probe until you are sure. But, do it neutrally. 

A statement like, "Then what you really mean is ..." does not convey 
neutrality. 

Sometimes in answering one question, the respondent will also 
answer another question. However, you should ask each question to 
be sure it is answered completely. However, do this in a way that 
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acknowledges their earlier comments. Probe especially on open 
ended items that have been partially answered in response to other 
questions. For example, you might say: 

I think an earlier comment you made might have answered 
some of the next question. Let me read the question and my 
notes on your response, and then you can clmfy or add to the 
comments* 

Do DfiLsynthesize an answer to a question out of their earlier 
comments when they have not spNwafically addressed the content of 
a question, even when. Aeir position is obvious to you. However, 
for questions with specific multiple-choice answers, their earlier 
answers may clearly answer these questions too. In cases where 
they have in effect given nearly verbatim responses you may use the 
following statement 

You might have already answered the next question. It asks 
[read the question], and you said [read recoraed answer]. Is 
that correct? 

Before accepting an answer of "I don't know," be sure to probe. 
Respondents frequently use that phrase in a way that says, "Fm 
thinking!" 

Some example of probes you might use: 

Yes, I see, (or) Uh-huh (stated in an expectant manner and 
followed by a pause) 

Could you be a little more specific? 

I am not sure I am entirely clear about what you mean. Could 
you explain it a little more? 

Could I read back what I have written down to be sure I have 
exactly what you want to say? 

DO NOT agree with answers the respondent gives. 
DO NQI EXPRESS YOUR OPINIONS regarding the economics 
education curriculum, the inadequate funding of education, politics, 
the weather, or any other topic. This is a vital difference between 
a neutral probe and a biasing probe. 

Write down everything. 

If a respondent qualifies an answer, or if a comment (probe) you 
offer stimulates a new response, write it down. Attempt to get it 
in verbatim form if it doesn't break the flow of the interview. You 
may need to rewrite answers after interviews to expand out what 
was said. Take good enough notes during the interview to dlow you 
to accurately expand the answers later. 
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6. 



7. 



Before you hang up scan over the form to be sure you have asked 
all of the questions for each project mentioned. 

If the length of the survey causes them to want to terminate in the 
middle, ask if you can call back to finish it As a last resort, ask 
who else in their office you might talk to. 



D. When you hang up: 



2. 



Inamediately note time and calculvite length of interview. Record 
this on the survey cover sheet 

Immediately go over every single answer to make sure it was done 
correctly. Fully expand out notes for answers. Rewrite answers 
which you even suspect might be illegible. 



E. At the end of each work day: 

1. Fill out the log of time and calls you have made for this survey. 
Record the long distance time used. 

2. Call Bill Quinn to report the day'5> work and discuss any problems or 
concerns. If a big problem comes up during the day don't hesitate 
io call me. 



Possible Answers to Refu sals for Economics Education Surv ey 
REASON FOR REFUSING . . . AND POSSIBLE RESPONSES 



TOO BUSY 



NOT INTERESTED 



OBJECTS TO SURVEYS 
OR TO PHONE SURVEYS 



This should only take a short time. Sorry to have 
caught you at a bad time. I would be happy to call 
back. When would be a good time to call in the 
next day or two? 

Its extremely important that we get accurate 
information about each of the districts otherwise the 
results won't be very useful. So, Td really like to 
talk to you. 

We think this particular survey is very 
interesting because the questions deal with an 
in^rtant issue in Utah education. We are doing 
this survey by telephone because this way is so much 
faster and it gives you a better opportunity to 
express exactly what you want to say. 
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What Thev Might Like to Know About Economics Education Sur vey 

WHAT IS THE WASATCH INSTITUTE FOR RESEARCH AND EVALUATION? 

It is a consulting firm which was established to provided research and 
evaluation services concerning problems in education. 

WHO IS SPONSORING (PAYING FOR) THE SURVEY? 

The Jordan School District, as part of their grant from the Utah State 
Office of Education, has commissioned this study to aid in planning future 
curriculum projects. 

WHAT IS THE PURPOSE OF THIS SURVEY? 

This information will aid the Utah Office of Education in planning future 
curriculum projects. 

WHO IS THE PERSON RESPONSIBLE FOR THE SURVEY? MAY I TALK TO 
HEM? 

The person is Dr. William Quinn. I am sure he would be happy to talk to 
you. I can have him call you, or if you like you can call him collect. 

HOW DID YOU GET MY NAME? 

Your name was on a list of those attending the Brian Head workshop. 

IS THIS CONFIDENTL\L? 

We will not be asking any personal information of you. The survey will 
only deal with you opinion of the economic education material, die 
training provided to help teachers use it, and your districts use of these 
materials. Your name \wll not appear in the report, and only a summary 
description of the professional positions of respondents will be reported. 

CAN I GET A COPY OF THE RESULTS? 

Yes, we would be glad to send it to you, if you will give me your current 
address. We hope to have the results ready in about four months. 
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Attachment 3: Instructional Design Review by Dr. Gale 
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Utah Economics Curriculum Materials: 
Comments on the Instructional Design 
fotroduction 

In the following sections the instructional design and instructional 
strategies used will be commented on based on the objectives for the program 
and the types of learning required for those objectives. The developed 
materials will be briefly described from this perspective with general comments 
on the appropriateness of the strategies used Specific comments will also be 
made about each group of lessons. A final section wili make recommendations 
for the incorporation of valid instructional design strategies in the team 
development of instructional materials. 



What Was Planned Or Intended 

Objectives 

The intentions of the economics content provided by the state appear to 
be to teach elementary and secondary students to both understand and 
appreciate (be positively inclined towards) the free enterprise system. This 
suggests that a major reason for the State-prescribed content is to persuade 
students to have a positive attitude toward the free enterprise system, to 
motivate them to support the system, and not just simply understand it 
However, the objectives actually used in the materials are, without exception, 
descriptive of a type of learning referred to in the literature as die learning 
of defined concepts. Those developing the materials dutifiilly followed the 
objectives and developed materials which, for the most part, encourage the 
acquisition of defined concepts only. This is problematical if persuasion and 
motivation were intended, as proven instructional strategies for motivating or 
persuading have not been included and will only occur by accident 

The Perspective Used 

Different types of learning can be identified as being unique, distinct 
from each other in that each learning type requires the use of different 
instructional strategies. According to one interpretation of research over the 
last two decades, there are at least nine different types of learning, each of 
which requires that different conditions be met and tiiat different elements be 
present in the instruction for each learning type. Such a perspective suggests 
that prescription is possible and that the findngs of educational research can 
be employed to produce better products. While such a position does not argue 
against intuitive insight or experience, it does suggest that there is a 
scientific base from which we can work as we develop instructional materials 
and that that base is a more certain foundation upon which we should build 
while being artistic as we apply principles that research suggests are 
beneficial. 
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Consistent with this t>crspcctivc, learning types can be categorized 
according to the following areas: 



Figure 1: Learning Types 

Concrete Concepts 

Discrimination & Paired-Associate 

Defined Concepts 

Abstract Concepts 

Rule Learning and Rule Use 

Problem Solving 

Psychomotor Skills 

Attitudes (Motivation, Persuasion) 

(Verbal Learning)* 

Verbal learning is not really a learning type but a way of teaching. It is 
included because most instruction takes tMsform. 



For each of die above learning types a different strategy should be used 
Research tells us diat different conditions need to be met and different 
elements need to be present if we are to be successful with each learning 
type. For instance if we*rc dealing with learning discriminations we would 
develop courseware that satisfied the conditions listed in Figure 2. If, on the 
other hand, we arc teaching defined concepts we would want to consider the 
information in Figure 3- Oiteria extracted from research has been iden :ified 
for each of the learning types listed above but have not been included in this 
document 
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Figure 2: For Discrimination Learning 

Conditions to Satisfy ; 

T The learner needs to know the name, label, or function of the 
object. 

2. Present the objects with their names, labels, descriptions or 
demonstrations of their functions. 

3. Provide practice identifying each object. 

4. Provide selective reinforcement of correct vs. incorrect responses. 

Elements that Need to be Prese nt; 

Multiple opportunities for practice 

2. Increasing levels of difficulty (possibly) 

3. Distractors normally found in context that may cause confusion 

Examples ; 

1. Learning the letters of a new alphabet like Russian or Japanese 

2. Learning to distinguish between differing shades or tints of the same 
color 

3. Learning to identify the taste of real vanilla extract 

4. Learning to identify the sounds of Morse Code letters 

Comment s; 

This learning type is usually pre-requisite to most other forms of learning 
but is so simple it is generally overlooked. 



F'^re 3; Teaching Defined Concepts 

Conditions to Satisfy ; 

1 . Learners must know the vocabulary used in the definitions. 

2. Learners must know any component concepts. 

3. Learners must be able to distinguish the subject from its 
relationships. 

Elements that Need to be Prese nt; 

1. Presentation of the concept and its definition 

2. Provision of examples and appropriate distractors 

3. Provision of ample practice opportunities 

4. Provision of prompt feedback to reinforce correct choices 

5. Final evaluation of learners* abilities to identify arui use the concept 

Examples ; 

1 . Teaching the concept of prime numbers 

2. Teaching the concept of laser light used for welding 

3. Teaching the concept of "supervision'' 

4. Teaching the concept of measurement (not evaluation) 

Comments; 

Because we rely so heavily on verbal learning this learning type is, by 
far, the most common. However, most instruction doesn' t provide the 
necessary examples, ample practice or feedback. 
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Types of Learning Required for the Economics Objectives 

As indicated above, the objectives suggest that all learning will be 
couched in the form of defined concepts. However, implied in these objectives 
as a whole, is the intent to ijersuade students that the btc enterprise system 
is something that is beneficial, woiks well and should be maintained Also 
implied by some of the objectives is a fomi of learning called paired associate 
learning where labels or terais need to be matched with appropriate definitions. 
While the types of learning actually required of students will vary from 
classroom to classroom and depend on the teacher^s spwific application of the 
materials provided, it q)pears from the lesson plans available, that defined 
concepts, and limited pjMtd-associate learning are the types of instruction 
planned for students. 



General Reaction to the Developed Materials 
Apparent Strategies Used 

Depending on the team member working on the materials, unique and 
creative approaches to teaching the content were used that are certain to 
capture the attention of at least some of the students* Simulations, class 
activities, and even field trips were recommended that were creative and 
beneficial. The large resource bank identified by teams developing the 
materials also provided a valuable tool to the teachers as a variety of different 
resources and media were identified and described for teacher and student use. 
(Obviously, the availability of such materials fixMn district to district would 
make a difference in how the instruction would be implemented; and a 
teacher's inclination to use or avoid the use of different media forms would 
also limit the effectiveness of that which is planned). By and large, those 
designing the lessons provided teachers with very practical suggestions and 
tools which would make it easy for the classroom teacher to quickly and 
simply adapt what is recommended to their particidar needs and circumstance. 
In this regard the materials have a particular strength and reveal die practiced 
experience of those who developed them. The suggestions for classroom 
activities, simulations, field trips, etc., were the real strength of the 
instructional materials. 

As suggested above, virtually all of the materials reviewed taught and 
tested for defined concepts only. Even the class activities, simulations, and 
field trips appeared to have this learning type as the only end result This is 
not surprising as most instruction in the public schools and higher education 
assumes that a student's ability to define something proves his or her ability 
to understand and apply it Unfortunately, this is an assumption proved 
erroneous by research. 

General Problems 

Problems occurring on a regular basis across grade levels and development 
teams included: 
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1) several objectives appeared to be very ambitious and the instructional 
materials, therefore, could not readily measure up to that which was required* 

2) Quite firequeady the classroom activities and the simulations that 
would be of most interest to the students also might be distractors in that 
they require students to pay more attention to the details of the simulation 
than the underlying principles being taught In many cases this problem could 
be rectified by changing the simulation slighdy or by constandy calling 
students' attention to why they're involved in that acti.vity and to pull die 
instruction together in a scries of review activities, tests, etc., that would help 
students focus on the principles rather than the game. 

3) Too firequendy dicre was an apparent mismatch between die objectives 
or goals stated and the specif}'^ classroom activities, simulations, or other 
suggestions provided It seemed like some developers created lesson plans, 
activities, etc., based on tilings that had worked for diem in die past in 
maintaining student attention and keeping students busy, but that these 
activities were not based on the stated goal or objective. 

4) Many of the materials used in classroom activities, and simulations 
especially, appear to be heavily borrowed from existing textbooks or odier 
manuals. Depending on how they are duplicated and used, some uses may step 
over die lines of propriety into copyright violation. Siuce so many of die 
activities and simulations that appear to be interesting and potentially powerful 
were borrowed from existing workbooks or texts and because diey may be used 
state-wide, their use could conceivably be a violation of copyright It would 
seem advisable to develop instnicrional materials based on such simulations, but 
to make diem substantially different in their detail or context so diat in diis, 
and possibly odier ways, die state could avoid potential confrontation or even 
litigation. 

5) Still another problem recurring across grade levels and development 
teams was die fact diat die quality and quantity of materials provided for 
each lesson varied substantially with the developer providing theoL Some of 
the teachers on the development team had a natural knack for identifying 
appropriate activities and resources, tywg these aU together, recommending 
how die teacher could use them while staying close to die original goal or 
objective. Others displayed a consistent difficulty matching activities with 
objectives, providmg enough content and substance to be helpful, and/or 
showing how the different materials and activities had any continuity. This, 
of course, is to be expected in any team effort but it would seem diat 
management procedures should early identify those who have the skills and 
keep them on the project while releasing those who have no such skills or 
inclination. 
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Comments Regarding Specifics 

Ihstractional Materials Developed for K To 6 Grade 

The project began somewhat tardily owing to a variety of reasons, and 
woric began in earnest in January 1986. All the work was based on what die 
end product should look like and this conceptualization of the end product has 
evolved to become something of a resource file and curriculum guide organized 
in a file drawer. As the system was conceptuaUzed, the district decided to 
include all subject matter content, not just economics, in the system, and 
much of the effort and energy was spent developing die system and organizing 
it for all content areas. Areas included in the system are geography, 
geography skills, economics, social studies, citizenship, career education, 
government, history, social skills and social systems. 

The conmiittee of two teachers from each grade level began their work 
hy visiting others and harvesting ideas for die teaching of economics. From 
this harvest they formulated strategies, wrote/created die materials, and then 
looked for and occasionally purchased printed and other media in support of 
what had been developed 

The entire system is based on concepts with each concept tied to an 
objective and where each concept has at least one, and typically two or three, 
suggested teaching strategies. Additionally, support materials and resource 
materials are recommended that are tied to the strategies. These materials are 
intended to help with terminology, vocabulary, provide references, tests, etc. 

The team members do not see the resulting materials as an end product 
but as a prototypes diey would like to continue refining. 

From the cvaluator's perspective, the resulting materials are the same as 
a district curriculum guide, only they are not bound in a manual but are 
carefully coded with colors and key words in a series of file folders. 

Specific comments regarding this set of instructional materials include: 

1) The "strategy sheets" should be carefully reviewed for basic spelling 
errors. 

2) There were packets where; 

a) the objectives and activities didn't match very well, 

b) the activities actually distracted from die goals stated, 

c) diere doesn't appear to be any summation or review or 
activities to tie the different concepts together and encourage 
students to focus on what they are supposed to notice and 
retain. 

3) Some of the instructional materials appeared to be incomplete and 
others were anemic, providing very little content or suggestions for 
activities. 

4) Virtually all of the materials assumed the learning type was defined 
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concepts and expected students to read, to listen to lectures and to 
cither verbally respond to or write about the defined concepts 
involved. 



5) Where strategics required students to actually apply the concepts to 
rcal-lifc situations, they appeared more likely to attract and 
maintain smdent interest 

6) An occasional simulation was created, but they were not much in 
evidence in this set of materials. 

7) There was a wide variance in what was available for economics 
through the grade levels, where some grade levels seemed to be 
reasonably complete and others had anemic lessons sporadically 
scattered through the objectives. 

8) The box of instructional materials reviewed consisted of at least 
two-thirds other subjects and less than one-third economics. This 
can give a casual reviewer an inflated or false picture of what's 
available - of what's actually there for the economics curriculum. 

9) The dissemination of the materials appeared to have substantial 
problems primarily because those responsible for dissemination were 
not involved early in the development of the materials and didn't 
have much ownership of thenu 

10) Also, the idea of having an inscrvice as the primary dissemination 
strategy would not appear to be as worthwhile as a combination of 
an inservice followed by team members.taking the materials "on the 
road" to display and demonstrate them in different schools. 

1 1) Tlie instructional materials were very similar in design, quality and 
general approach to the materials developed for grades 7-12. 

12) There was a very uneven quality in the materials and the activities 
or resouices proposed by them. 

13) The materials obviously had a strategy emphasis rather than an 
objective, content, or "what do we want the student to leam" 
emphasis. 

14) These materials were very simple and existed at a low level 
intellectually, we assume to match the grade level and interest of 
younger students. However, they tended to be too simple especially 
in the upper grades. A question could be raised about how 
challenging the materials are to the students. 

15) Occasionally the materials were tied to practical situations and 
direct applications to things with which the sr^dents are familiar - 
a very good idea. 

16) An analysis of the time and effort put into the development of these 
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materials compared with what was produced would likely suggest 
that development team members have not had the kind of experience 
that would make them efficient developers. 

17) A concern: How will continuity be maintained when the money set 
aside for development is gone and those who developed the materials 
leave (probably by this June)? Will the materials be continued and 
will they be used in the absence of those who arc attempting to 
disseminate and "sell" them and when no one really has ownership of 
their continuation? If this is a "one shot" project, then it falls mto 
the trap of projects m other states and districts where materials are 
shelved as soon as the development team goes away. 

Listnictional Materials Developed for Grades 7-12 

Rather than repeating most of die comments listed above for die 
materials developed for K-6, it should be observed tiiat virtually all earlier 
concerns apply to tiiese materials as well. 

A few writers on this team appear to have more experience, or natural 
talent, as evidcncwl by die practical application's and sunulations diat wiM. 
attract and maintain student mterest TTbe deveL >pers on this team appeared 
better able to coordinate the resources, activities, lecture suggestions, etc., 
widi die goals or objectives. The overall quality in diesc materials appears 
better. 

Specific comments for die 7 - 12 materials are used to emphasize their 
particular visibility in this set of materials. 

1) The in-classroom techniques and strategies recommended were, 
without question, die best part of die materials reviewed 

2) Virtually all of die matf,rials assumed diat defined concepts were 
being taught, tiiat they were die key to "understanding" economics 
and the free enterprise system. 

3) Some of die simulations appear to have or be distractors with 
students paying more attention to die game itself rather than die 
pruicirles it was supposed to demonsttate. Again, die solution is to 
not throv/ die game away but rather make a studied attempt to 
focus on the principles being treated 

4) Materials were regularly not created but heavily borrowed from 
existing, and probably copyrighted, works. 

5) The quality of the materials varied substantially from writer to 
writer widi some displaying a natural talent for development. 

6) Occasionally the reader is uncertain as to whether portions of a 
course are being developed or sunply a showcase of useful materials. 
This probably stemmed from die fact diat die recommended media 
were not available to review and it was not obvious how these might 
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fit into the suggestions and lessons planned. 



7) The instructional materials developed were generally aimed at 
teacher use only and didn't emphasize what was expected from 
students. While there is an obvious relationship between the two, 
this focus on the teacher is usually detrimental to the objectives 
and goals intended 

Again, the concerns listed above in the K-6 materials repeated themselves 
in this set of materials and the numbered items listed above, where they are 
duplicates of the earlier concerns, arc statod for emphasis - to show which 
concerns recurred frequently in this set of materials. 



Recommendations for Next Time 

The Development Team 

We strongly recommend that a team be used m the development of 
instructional materials. However, how the team is selected and who actually 
serves on the team ought to be controlled for beneficial effect It would seem 
wise to include on that team an individual well-trained in both the 
insmictional design process and instructional design principles. While an 
individual trained in the traditional "curriculum and instruction" programs could 
also be useful on such a team, the training we're referring to woiUd include 
practical experience in the application of research results (instructional design 
principles) to the creation of mstructional strategies and materials. While the 
label of that person or the program ftom which they £jaduated varies across 
the U.S., die point is that die individual would need to know both the process 
of developing materials and educational research results so s/ht could apply 
the same beneficially to the materials created 

It would be useful to have someone well schooled in evaluation (not 
student testing alone) on the team. An experienced evaluator who would 
challenge design decisions early in the project could have a positive, refining 
effect on the entire process, not just on the formative evaluation that occurs 
relatively late. 

Obviously, developers (writers) would also be essential on the team. It 
would seem best to recruit the majority of these writers from the classroom, 
but in order to maintain the level of quality across the materials developed, it 
is also necessary to identify teachers who have experience or a natural talent 
for writing and development A process for accomplishing this that has been 
used successfully in the past includes holding a competition where teachers are 
provided with a general design document with objectives and invited to create 
instructional materials, strategies, classroom activities, etc., appropriate to the 
design description. Besides creating useful materials in the competition, the 
best of the developers/writers could be identified by the project director, 
evaluator, instructional designer, and put to work creating the course. 

Once you've identified those who have a knack for creating sound 
instruction, it becomes possible to assign the best of them to direct elements 
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of the project or to serve a5 quality controllers for what others are producing. 

Normal business practices should be expected in the management of the 
team and individuals not producing quality materials or meeting deadlines 
should understand that they can be released and others retained in their stead. 
Choosing people to be members of a team based solely on the rcconmiendations 
of others and then hiring them to the team on a permanent basis would clearly 
be an unwise way to proceed where consistent quality and substance in the 
instructional materials are desired outcomes. It woidd be inappropriate for 
extra curricular activities and associated pay to be passed out as rewards to 
people that have performed well in the classroom but who may not be the best 
mstmctional developers or writers in the district 

The Instractional Development Steps 

Steps used in the instructional development process are generally well 
known but occasionally used. They include: 

1) Need assessment 

2) Task and/or content analyses 

3) Researching available resources and references 

4) Instructional system and strategy design 

5) Development of the instructional materials 

6) Formative evaluation of the resulting materials 

7) Revision of the materials, strategy, or system based on the formative 
evaluation results 

8) Iii.^.ementation of the course 

9) Summative evaluation of the course 

In addition to the above steps we reconmiend that development teams use 
what we refer to as fast prototyping. That is, we recommend that a strategy 
be developed consistent with learning types identified in the content/task 
analyses and that its processes be immediately tested in different school 
settings with two or fhree different lessons. Information collected from the 
formative evaluation of those tests would likely alter the strategy requiring 
that it be revised, wtested or scrapped After these quick tests, and only 
after such tests, should a full set of instructional materials be produced using 
that strategy. Once a strategy has successfully survived the prototype testing 
and a full course of materials is developed (using all strategies similarly 
tested), the formative evaluation for the course ^oiild be conducted and will 
suggest still other ways that the strategies can be refined or revised. Budgets 
in support of instructional development should include support for the 
necessary prototype evaluations. 

An important and potentially very uceful step early in the instructional 
design process is the collection of ideas from classroom teachers who have 
direct experience teach the targeted subject matter at the appropriate ^ade 
level. This k just a "harvest" of ideas, however. To accept the id^^as as-is 
without running them through the instructional design document or without 
considering how carefully such ideas match objectives and the instructional 
strategies that should be used for specific learning types would be a serious 
mistake. 
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Phase- Wise Development 



It would seem both wise and practical for the state to insert itself into 
the development process by requiring those who create materials to submit, for 
approval and continued funding; early design strategies, the instractional design 
document, initial prototype lessons, and prototype evaluation means. As each 
of these products is reviewed (preferably after testing in the classroom), the 
next stej^s could be approved and funding tied to that phase released to the 
contracting district TTiis is how many businesses operate their Research and 
Development processes and it would appear wise for the state to encourage 
qualiQT control by applying similar tact7C5. 

Such an approach would encourage the school districts even more to use 
the best team members on the project; to hire qualified evaluators or 
instructional developers if they don't have them available in their district and 
to, in other ways, be quality conscious. Also, requirir . , the careful formative 
evaluation of two or three prototype lessons would teach team members 
invaluable lessons about what wonted, what didn't, and more importantly, why. 
Spending money to develop literally dozens of lessons without trying them out 
in the classroom is not a wise use of resources. While there is always 
pressure to develop and get the materials out as quickly as possible, we should 
remember the familiar lament: "Why is there never time to do it right, but 
always time to do it over?" 

Developing A Design Document 

An instructional design document is provided by way of example. Such 
documents are drafted to provide direction, descriptions of contexts, and 
delimitations to the instructional design process. While it's critical that 
writers not overlook steps in the instructional design process, it's just as 
critical to carefully conceptualize the limits and flavor of the course being 
developed so that writers operate more as a team and produce materials whose 
continuity is both obvious and consistent This can be accomplished through 
such a document 

The elements in the following instructional design document are provided 
as a "straw man" to work from and their function is either demonstrated with 
a partial example or through narrative that describes what that section is 
supposed to delineate. 
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Figure 4: An Instructional Design Document Example 

The following categories might be considered for inclusion in an original 
design document that would detail the specifics of a state planned course and 
show how the specifics are needed given the context of the need, the intended 
audience and the types of learning required. Such a design document ought to 
be a part of the instructional development approval process used to insure 
appropriate content, scope, treatment of the content and methods used in the 
course. After most of the categories a very brief example is provided by way 
of clarification, or a simple narrative describes what ought to appear in that 
section. 

CONTEXT DESCRIPTION 

The Audience 

The audiences for which the course is bein? developed are secondary 
students of both junior high and high school. It is assumed that the normally 
hroci] distribution of interests and abilities will exist among the students and 
that a percentage of them wili be negatively predisposed against . . . 

The Delivery System 

The system that will communicate information and provide students with 
the opportunity to acquire skills in "X" will include (list of equipment, books, 
technologies to be used). The design of the delivery system will integrate 
(books, methods, technologies) into the normally functioning classroom and will 
take advantage of the expertise of the classroom teacher as a motivator, 
diagnostician/prescriber, evaluator, and instructional manager. 

Resource Documents 

Student supplementary materials, teacher's support materials and 
documents describing how to use the course and equipment system will 
accompany the course. 

Course Goals 

The goals of the course are listed by Areas, Umts and Lessons: 

(These would then be listed) 

THE INSTRUCTIONAL DESIGN 

Learning Types and Strategies 

A practical approach to designing instruction requires that the learning 
types required of the students be identified so ihat the conditions that need in 
be satisfied and the elements that need to be present for each learning type 
. will be incorporated into the instructional materials and system. The learning 
types required of the students that have been identified for :he jourse, so far, 
include: 
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* discrimmtion 

* concrete concepts 

* defined concepts 

* rule learning 
abstract concepts 

* motivation or persuasion (concerning the need to understand and 
useXYZ). 

* problem solving (a possible, infrequent learning type during the 
first semester) 

(Another list of learning types would be provided for each new course. Some 
prefer to do these analyses by unit or lesson.) 

General Presentation Strategy 

Video will be used as the major presentation medium either as circulated 
video tapes. As the bulk of the course will be dealing with discrimination 
learning and defined concepts, the general instructional strategy used will be: 

1) provide an object or situation with its appropriate 
name/label/definition. (Or present the concept or rule with its 
description.) 

2) supply examples and possibly non-example distractors of each, 
(object, concept or rule) 

3) provide ample and multiple opportunities for students to practice 
their new htowledge; 

4) supply rapid and appropriate feedback based on their practicing; 

5) test student knowledge/abilities. 

Motivation and persuasion techniques will be imbedded throughout the 
course in the way that content is communicated, in the situations, stories and 
examples used to tie the unfamiliar and new back to the students' interests 
and e:q>erience, and in the games that will be ckveloped on CAI to provide 
simula'^'^d practice. 



COURSE DESIGN 

Each lesson will be composed of five segments: 1) The introduction, 
which has as its purpose changing the atmosphere and as an advanced 
organizer, showing each lessons' activities and content; 2) the relaxation 
session, intended to move students out of what they see as the traditional way 
of learning to something more cornfortable; this segment will also be used as 
an advanced organizer or for review of older terms and concepts; 3) a review 
arui presentation segment where earlier words and phrases are reviewed and a 
limited amount of new content is introduced and carefully controlled; 4) the 
passive practice sessions where students are encouraged to...; and 5) the 
practice sessions which consist of a variety of activities designed to provide 
the student with different opportunities to practice new concepts. 

In all of the above activities, exposure to new concepts and ideas will be 
very carefully controlled and will be orchestrated with meticulously tracked, 
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integrated reviews. 



LESSON DESIGN ELEMENTS 

Each lesson will be comprised of the elements listed below with each 
lesson being obviously different in its sequence of instruction, examples used, 
etc. so that the format doesn't become tiring. 

The course is assumed to lastX weeks with Y minutes of instruction and 
practice occurring daily. The course will be broken down into ''units" of 3-5 
days duration and each unit will be somewhat self contained in its stricHy 
controlled use of new terms, new concepts and "inntexts. Each unit will be 
built iq)on a context that stays constantforthmt nit; contexts such as: the 
family; dating; sports, etc. There will be a total of 16 to 26 different 
contexts that will systematically incorporate expropriate . . .(content related) 
information. 

With the beginning of each new unit new . . .(content related) will be 
introduced. Previously introduced information will be systematically reviewed 
throughout the course with new terms reoccurring 7 or more times and 
concepts being repeated many times each. 

In order to build self coi^dence, the course will be designed to provide 
maximum support and positive reinforcement to students especially during the 
first weeks. Efforts will be made to help students better tolerate the 
uncertainty and ambiguity of. . . Feedback to the students will be positive 
even when correcting their misapplied efforts. 

Specific Elements of Each Lesson 

Each lesson will contain the following elements: 

L Classroom and lesson preparation on the part of the classroom 
teacher 

2. The presentation, broken down into: 

a. The Introducticn 

b. Relaxation 

c. Presentation and Review 

d. Active Practice 

e. Passive Practice 

f. Application (Recognizing and Using Ne^v Knowledge) 

3. Classroom Practicing 

a. CAI lessons/practice 

b. gantes 

c. recitation 

d. group study 

e. paired study 
^ / singing 

g. . . . 
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4. Practice at Home 



The Purpose of Each Lesson Element 

The Preparation segment of the lesson requires the classroom teacher to 
prepare the classroom with expropriate props, posters, etc. and to review the 
kinds of games and classroom activities that sihe will be using for that lesson. 

The lesson starts with the Introduction which is to be used to encourage 
the students to ''shift gears", calm down and focus on . . . The classroom, its 
posters, fijumiture, props, music, etc. are all calculated to create a pleasant 
atmosphere. 

The Presentation and Review section presents new information in a 
context and reviews previously presented . . .(content related). When a 
unit is introduced, this section serves as a ''map'' of what wiV be dealt with 
during the next three to five lessons. Subsequent lessons in same unit 
will repeat this new ir^ormation and continue to review earlier lesson skills 
and content. 

Active Practice requires the student to... 

Passive Practice is . . . 

The Application element of the instruction is intended to provide students 
with the opportunity to test their understanding and use of the new knowledge 
and skills presented. 

The Classroom Practice could consist of virtually anything that will 
encourage students to use their newly acquired knowledge and skills. 

Lesson End Products 

The lessons will be made up of: 

a. The classroom props, posters, and relaxing environment 

b. The classroom teacher^ s materials (teachefs manual) 

c. The lesson 

d. The CAI games, lessons and data bases 

f. Classroom games and activities 

g. Recommendations for practice at home 

h. Classroom management software, evaluation forms, and suggestions 



SCOPE AND SEQUENC7, 
The scope and sequence of the course. . . 
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INSTRUCTIONAJ. DESIGN CRITERIA 



Conditions to Satisfy 

For each of the learning types listed earlier in this document, specific 
instructional design elements and criteria will be inco porated by developers, 
writers and producers. (An example of the instructioml conditions, elements 
and criteria for one learning type is provided). 

Discriminations ^ The conditions to satisfy in learning discriminations are: 

1. A pre-condition: The learner needs to know the name, label, or 
fimction of each object to be discriminated. 

2. Provide the objects with their names, labels, descriptions or 
demonstrations of their functions. 

3. Provide practice identifying each object. 

4. Provide selective reiftforcement of correct vs. incorrect 
responses to the object. (Negative feedback should provide the 
correct answer or a clue to the correct answer and should not 
intimidcte the student with his incorrectness.) 

The elements that need to be present include: 

1. Multiple opportunities for practice 

2. Possibly increasing levels of difficulty 

3. Distractors, normally found in context, that may cause 
confusion or interference. 

(How this would then be incorporated into the course would then be 
ejq>lained). 



MESSAGE DESIGN CONCERNS 

(Merzage design principles should be paid attention to for all learning 
types. Rather than mentioning all possible message design principles related to 
this course and its delivery system, only a sample of the major concerns will 
be detailed. This series assumes that lessons include CAI elements). 

All video screens and CRT "pages" will follow the "keep it simple" 
principle of uncluttered information with ample margins and white space. 
Texture, color, grouping, dimension/scale, shape and tone will be used to tie 
like elements together, to attract student attention or cue them to notice 
critical information. 

Message Design Checklist 

(A sampling of message design concerns that applied to this project 
included:) 

* Do the instructional materials help learners to selectively attend to the 
desired information/message? 
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Are cues, schema, visuals and so forth used to help the learner's storage, 
search and retrieval str^.tcgies? 

How are the pieces of content (facts, labels, concepts) related to each 
other? Does the organization of content (or of the lean mig experience) 
show these relationships and thereby promote retention}' 

Do the foreground graphic or text colors avoid the use of red or blue? 

Are the text columns si^ciently narrow to permit (q)propriate eye 
scanning patterns? ■ 
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